How do social scientists collect data
to answer the questions?

• We then have to resort to a subset of units that we can reasonably collect data for \(\Rightarrow\) Sample

  • Example: Those who participate in the survey; News reports that we can download online

• We collect the sample from the available list that ideally includes the whole population \(\Rightarrow\) Sampling frame

  • Selection bias: list of registered voters (frame) might include nonvoters!

• Simple random sampling: Every unit has an equal selection probability

  • e.g. random digit dialing (RDD):

    1. Take a particular area code + exchange: 617-495-XXXX.
    2. Randomly choose each digit in XXXX to call a particular phone.
    3. Every phone number in America has an equal chance of being included in sample.

• Quota sampling, cluster sampling, etc.

• Non-probability sampling: e.g. Opt-in Internet panels

  • Question: Can you see a problem with this?

• George Gallup sampled only 50,000 respondents from all voting age citizens

• Ballots skewed toward the wealthy (with cars, phones) \(\Rightarrow\) selection bias

  • Only 1 in 4 households had a phone in 1936.

• People who respond could be different than those who don’t \(\Rightarrow\) nonresponse bias

• Note: When selection procedure is biased, adding more observations doesn’t help!

• Quota sampling:

  • fixed quota of certain respondents for each interviewer
  • sample resembles the population on these characteristics

• Potential unobserved confounding \(\Rightarrow\) selection bias

• Republicans easier to interview within quotas (phones, listed addresses, etc.)

• Descriptive (summary) statistics are numerical summaries of those measurements

  • If we smart enough, we wouldn’t need them: just look at the list of numbers and completely understand

• Two salient features of a variable that we want to know:

  • Central tendency: where is the middle/typical/average value
  • Spread around the center: are all the data close to the center or spread out?

• Mean: sum of the values divided by the number of observations

\[ \color{#98971a}{\bar{x}} = \color{#d65d0e}{\frac{1}{n}} \color{#458588}{\sum_{i = 1}^{n} x_{i}} \]

• What’s all this notation?

  • Population value: Greek letters
  • Sample value: Latin letters
  • Miscellaneous squiggles (sums, hats, bars, subscripts)

• Applied to the mean:

  • Population value: \(\mu\) (say mu)
  • Sample value: \(\hat{\mu}\) (say mu-hat); \(\bar{x}\) (say “x-bar”)

• Median more robust to outliers:

  • Example 1: \(\text{data} = \{ 0, 1, 2, 3, 5 \}\). \(\text{mean} = 2.2\), \(\text{median} = 2\)
  • Example 2: \(\text{data} = \{ 0, 1, 2, 3, 100 \}\). \(\text{mean} = 21.2\), \(\text{median} = 2\)

• Question: What does Elon Musk do to the mean vs median income? \(\Rightarrow\) income inequality measure

• Range: \(\left[ \min (X), \max (X) \right]\)

• Quantile (quartile, quintile, percentile, etc):

  • 25th percentile = lower quartile (25% of the data below this value)
  • 50th percentile = median (50% of the data below this value)
  • 75th percentile = upper quartile (75% of the data below this value)

• Interquartile range (IQR): a measure of variability

  • How spread out is the middle half of the data?
  • Is most of the data really close to the median or are the values spread out?

• One definition of outliers: over 1.5 × IQR above the upper quartile or below lower quartile

\[ \text{sd} = \color{#cc241d}{\sqrt{\color{#b16286}{\frac{1}{n - 1}} \color{#98971a}{\sum_{i = 1}^{n}} \color{#458588}{(}\color{#d65d0e}{x_i - \bar{x}}\color{#458588}{)^2} }} \]

• Steps:

  1. Subtract each data point by the mean
  2. Square each resulting difference
  3. Take the sum of these values
  4. Divide by \(n − 1\)
  5. Take the square root

• Variance (\(\sigma^2\), var): \(\text{Var} = \text{standard deviation}^2\)

• Question: Why not just take the average deviations from mean without squaring?

• Empirical analyses: sums or means of these n measurements

  • All statistical procedures involve a statistic, very often sum or mean.
  • What are the properties of these sums and means?
  • Can the sample mean of age tell us anything about the population distribution of age?

• Asymptotics: what can we learn as \(n\) gets big?

• Intuition: The probability of \(\bar{X}_n\) being “far away” from \(\mu\) goes to \(0\) as \(n\) gets big
• The distribution of sample mean “collapses” to population mean

• Three key properties:

  • Unimodal: one peak at the mean
  • Symmetric around the mean
  • Everywhere positive: any real value can possibly occur

• Approximation is better as \(n\) goes up \(\Rightarrow\) asymptotics

• “Sample means tend to be normally distributed as samples get large.”

  • We now know how far away \(\bar{X}_n\) will be from its mean!

• By empirical rule, sample mean will be within \(2 \times \sigma^2 / n\) of the population mean 95% of the time

• We usually only 1 sample, so we’ll only get 1 sample mean. So why do we care about LLN/CLT?

  • CLT gives us assurances our sample mean won’t be too far from population mean
  • CLT will also help us create measure of uncertainty for our estimates, standard error (SE):

  \[ SE = \sqrt{\frac{\sigma^2}{n}} = \frac{\sigma}{\sqrt{n}} \]

• Always keep records of every answer you receive!

  • Now can be collected and stored electronically (e.g., Qualtrics, SurveyCTO, even Google Forms)

• Sample size:

  • Small-\(N\): usually face-to-face and can be semi-structured
  • Large-\(N\): Rarely conducted by the researchers themselves (Why?) and has to be structured

• Context: Which mode is most common/appropriate among the study population?

• Depth/Length: Is this a short factual survey or in-depth interview

• Sensitivity: Do you ask questions that require trust between researcher and respondent

• Logistics: What is the easiest/cheapest way to reach the respondents

• Security/Privacy: Are there concerns about using specific mode of interview?

• Running record: Materials that are collected systematically across time

  • Examples: Statistical agencies, databases with indicators, systematic geographic data, etc.

• Episodic records: Records produced in casual, personal, and accidental manner

  • Example: Diaries, personal blogs, social media posts

• Accessibility: Do you need to collect/process data from data source?

• Coverage: Are there systematic biases in data available?

• Interpretation: Is primary data/methodology available? Can you infer it?

• Direct observation: Observing the political behavior itself

  • Example: Laboratory observation of group interactions, observation of worker movement when working at the factory, browsing history of participants, etc.

• Indirect observation: Observing physical trace of the political behavior

  • Example: Political protests in factories (political mobilization), improvements in services provided to community (collective action)

• Research Question: Is pure observation of behavior enough? What about causal explanations?

• Observer effect: Does your observation interfere with how subjects behave?

• Safety: Does political behavior you study pose danger to you or enumerators?

• Privacy/Anonymity: Is your research posing any risk of harm or could implicate participants?

• Key examples:

  • Negative repercussion from associating with the researcher because of researcher’s sponsors, nationality or status
  • Invasion of privacy
  • Stress during the research interaction
  • Disclosure of behavior or information causes harm to observed during or after the study

• Also to consider:

  • Autonomy: ability to consent, withdraw, etc.
  • Justice: distribution of benefits and burdens